{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "93e3f84f",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/node_postprocessor/LLMReranker-Lyft-10k.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "11b52622-f38a-4b3a-a916-c73619babb48",
   "metadata": {},
   "source": [
    "# LLM Reranker Demonstration (2021 Lyft 10-k)\n",
    "\n",
    "This tutorial showcases how to do a two-stage pass for retrieval. Use embedding-based retrieval with a high top-k value\n",
    "in order to maximize recall and get a large set of candidate items. Then, use LLM-based retrieval\n",
    "to dynamically select the nodes that are actually relevant to the query."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91b61e96-864c-4ed2-80d6-0ebfdbb57d5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import nest_asyncio\n",
    "\n",
    "nest_asyncio.apply()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f630346f-fedd-40f2-8fb3-e052e219a873",
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "import sys\n",
    "\n",
    "logging.basicConfig(stream=sys.stdout, level=logging.INFO)\n",
    "logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))\n",
    "from llama_index import (\n",
    "    VectorStoreIndex,\n",
    "    SimpleDirectoryReader,\n",
    "    ServiceContext,\n",
    ")\n",
    "from llama_index.postprocessor import LLMRerank\n",
    "\n",
    "from llama_index.llms import OpenAI\n",
    "from IPython.display import Markdown, display"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "231c4418",
   "metadata": {},
   "source": [
    "## Download Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36f2294f",
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p 'data/10k/'\n",
    "!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/10k/lyft_2021.pdf' -O 'data/10k/lyft_2021.pdf'"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "8a8fdeb2-939c-49dd-b8e6-0139d53a4fb6",
   "metadata": {},
   "source": [
    "## Load Data, Build Index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee132733-b525-4aaf-81db-7eed19ded815",
   "metadata": {},
   "outputs": [],
   "source": [
    "# LLM Predictor (gpt-3.5-turbo) + service context\n",
    "llm = OpenAI(temperature=0, model=\"gpt-3.5-turbo\")\n",
    "\n",
    "chunk_overlap = 0\n",
    "chunk_size = 128\n",
    "\n",
    "service_context = ServiceContext.from_defaults(\n",
    "    llm=llm,\n",
    "    chunk_size=chunk_size,\n",
    "    chunk_overlap=chunk_overlap,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88344ea2-540c-4b46-8a66-ed78870cb80a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# load documents\n",
    "documents = SimpleDirectoryReader(\n",
    "    input_files=[\"./data/10k/lyft_2021.pdf\"]\n",
    ").load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d478f416-8e85-49ae-9817-e4d78122a120",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 226241 tokens\n",
      "> [build_index_from_nodes] Total embedding token usage: 226241 tokens\n",
      "> [build_index_from_nodes] Total embedding token usage: 226241 tokens\n"
     ]
    }
   ],
   "source": [
    "index = VectorStoreIndex.from_documents(\n",
    "    documents, service_context=service_context\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7e3d5f23-dfcd-458d-a9d3-dd66de0ab054",
   "metadata": {},
   "source": [
    "## Retrieval Comparisons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "47480c6d-8914-4562-a789-fd53a99a7afb",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/1r/c3h91d9s49xblwfvz79s78_c0000gn/T/ipykernel_58458/2502541873.py:8: FutureWarning: Passing a negative integer is deprecated in version 1.0 and will not be supported in future version. Instead, use None to not limit the column width.\n",
      "  pd.set_option('display.max_colwidth', -1)\n"
     ]
    }
   ],
   "source": [
    "from llama_index.retrievers import VectorIndexRetriever\n",
    "from llama_index.schema import QueryBundle\n",
    "import pandas as pd\n",
    "from IPython.display import display, HTML\n",
    "from copy import deepcopy\n",
    "\n",
    "\n",
    "pd.set_option(\"display.max_colwidth\", -1)\n",
    "\n",
    "\n",
    "def get_retrieved_nodes(\n",
    "    query_str, vector_top_k=10, reranker_top_n=3, with_reranker=False\n",
    "):\n",
    "    query_bundle = QueryBundle(query_str)\n",
    "    # configure retriever\n",
    "    retriever = VectorIndexRetriever(\n",
    "        index=index,\n",
    "        similarity_top_k=vector_top_k,\n",
    "    )\n",
    "    retrieved_nodes = retriever.retrieve(query_bundle)\n",
    "\n",
    "    if with_reranker:\n",
    "        # configure reranker\n",
    "        reranker = LLMRerank(\n",
    "            choice_batch_size=5,\n",
    "            top_n=reranker_top_n,\n",
    "            service_context=service_context,\n",
    "        )\n",
    "        retrieved_nodes = reranker.postprocess_nodes(\n",
    "            retrieved_nodes, query_bundle\n",
    "        )\n",
    "\n",
    "    return retrieved_nodes\n",
    "\n",
    "\n",
    "def pretty_print(df):\n",
    "    return display(HTML(df.to_html().replace(\"\\\\n\", \"<br>\")))\n",
    "\n",
    "\n",
    "def visualize_retrieved_nodes(nodes) -> None:\n",
    "    result_dicts = []\n",
    "    for node in nodes:\n",
    "        node = deepcopy(node)\n",
    "        node.node.metadata = None\n",
    "        node_text = node.node.get_text()\n",
    "        node_text = node_text.replace(\"\\n\", \" \")\n",
    "\n",
    "        result_dict = {\"Score\": node.score, \"Text\": node_text}\n",
    "        result_dicts.append(result_dict)\n",
    "\n",
    "    pretty_print(pd.DataFrame(result_dicts))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b8bedc4f-444b-4233-9b72-728e3cfbe056",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 11 tokens\n",
      "> [retrieve] Total embedding token usage: 11 tokens\n",
      "> [retrieve] Total embedding token usage: 11 tokens\n",
      "> [retrieve] Total embedding token usage: 11 tokens\n",
      "> [retrieve] Total embedding token usage: 11 tokens\n"
     ]
    }
   ],
   "source": [
    "new_nodes = get_retrieved_nodes(\n",
    "    \"What is Lyft's response to COVID-19?\", vector_top_k=5, with_reranker=False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e85e656b-9377-4640-a10d-a6655afd82bd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Score</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.863554</td>\n",
       "      <td>Rentals. Further, COVID-19 has and may continue to negatively impact Lyft’s ability to conduct rental operationsthrough  the Express Drive program and Lyft Rentals as a result of restrictions on travel, mandated closures, limited staffing availability, and other factors relatedto  COVID-19. For example, in 2020, Lyft Rentals temporarily ceased operations, closing its rental locations, as a result of COVID-19. Further, while ExpressDrive  rental periods</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.854175</td>\n",
       "      <td>pandemic, including sales, marketing and costs relating to our efforts to mitigate the impact of the COVID-19 pandemic. Furthermore, we have expanded overtime  to include more asset-intensive offerings such as our network of Light Vehicles, Flexdrive, Lyft Rentals and Lyft Auto Care. We are also expanding the supportavailable  to drivers at our Driver Hubs, our driver-centric service centers and community spaces, Driver Centers, our vehicle service centers, Mobile Services,</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.852866</td>\n",
       "      <td>requested to quarantine by a medical professional, which it continues to do at this time. Further, Lyft Rentals and Flexdrive have facedsignificantly higher cos ts in transporting, repossessing, cleaning, and17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.847151</td>\n",
       "      <td>the transport ation needs of customers, employees and other constituents.• Grow Active Riders. We  see opportunities to continue to recoup and grow our rider base amid the continuing COVID-19 pandemic. We may make incrementalinvestments in our  brand and in growth marketing to maintain and drive increasing consumer preference for Lyft. We may also offer discounts for first-time ridersto  try Lyft or provide incentives to existing riders to encourage increased ride frequency. We</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.841177</td>\n",
       "      <td>day one, we have worked continuousl y to enhance the safety of our platform and the ridesharing industry by developing innovative products, policiesand processes. Business Lyft  is evolving how businesses large and small take care of their people’s transportation needs across sectors including corporate, healthcare, auto, education andgovernment.  Our  comprehensive  set  of  solutions  allows  clients  to  design,  manage  and  pay  for  ground</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "visualize_retrieved_nodes(new_nodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ba150e2-c4e7-4404-b8e1-1603c2b346d3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "> [retrieve] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 11 tokens\n",
      "> [retrieve] Total embedding token usage: 11 tokens\n",
      "> [retrieve] Total embedding token usage: 11 tokens\n",
      "> [retrieve] Total embedding token usage: 11 tokens\n",
      "> [retrieve] Total embedding token usage: 11 tokens\n"
     ]
    }
   ],
   "source": [
    "new_nodes = get_retrieved_nodes(\n",
    "    \"What is Lyft's response to COVID-19?\",\n",
    "    vector_top_k=20,\n",
    "    reranker_top_n=5,\n",
    "    with_reranker=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7541606f-6424-470b-987c-a986ac0a7cf8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Score</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.0</td>\n",
       "      <td>inunrestricted  cash and cash equivalents and short-term investments as of December 31, 2021, we believe we have sufficient liquidity to continue business operations and totake  action we determine to be in the best interests of our employees, stockholders, stakeholders and of drivers and riders on the Lyft Platform. For more information onrisks associated with the COVID-19 pandem ic, see the section titled “Risk Factors” in Item 1A of Part I.Recent Developments Transaction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10.0</td>\n",
       "      <td>COVID-19, may continue to develop or persist over time and further contribute to thisadverse effect. • Changes in driver behavior during the COVID-19 pandemic have led to reduced levels of driver availability on our platform relative to rider demand in certainmarkets.  This imbalance fluctuates for various reasons, and to the extent that driver availability is limited, our service levels have been and may be negativelyimpacted  and we have increased prices or provided additional incentives and may need to continue to do so, which</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>estimated.In  response to the COVID-19 pandemic, we have adopted multiple measures, including, but not limited, to establishing new health and safety requirements forridesharing  and  updating  workplace  policies.  We  also  made  adjustments  to  our  expenses  and  cash  flow  to  correlate  with  declines  in  revenues  including  headcountreductions in 2020. 56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10.0</td>\n",
       "      <td>opportunities for drivers on our platform. Our business continues to be impacted by the COVID-19pandemic. Although  we have seen some signs of demand improving, particularly compared to the demand levels at the start of the pandemic, demand levels continue to beaffected by the  impact of variants and changes in case counts. The exact timing and pace of the recovery remain uncertain. The extent to which our operations will continueto  be impacted by the pandemic will depend largely on future</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10.0</td>\n",
       "      <td>does not perceive ridesharing or our other offerings as beneficial, or chooses not to adopt them as a result of concerns regarding public health or safety, affordability or forother  reasons, whether as a result of incidents on our platform or on our competitors’ platforms, the COVID-19 pandemic, or otherwise, then the market for our offeringsmay  not further develop, may develop more slowly than we expect or may not achieve the growth potential we expect. Additionally,</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "visualize_retrieved_nodes(new_nodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb88c0bb-4d3f-4426-b2be-66b0d5635abb",
   "metadata": {},
   "outputs": [],
   "source": [
    "new_nodes = get_retrieved_nodes(\n",
    "    \"What initiatives are the company focusing on independently of COVID-19?\",\n",
    "    vector_top_k=5,\n",
    "    with_reranker=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c4b3962-2873-40b3-9f50-58cf7685454a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Score</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.819209</td>\n",
       "      <td>businesses to contain the pandemic or respond to its impact and altered consumer behavior, amongother  things. The Company has adopted a number of measures in response to the COVID-19 pandemic including, but not limited to, establishing new health and safetyrequirements  for ridesharing and updating workplace policies. The Company also made adjustments to its expenses and cash flow to correlate with declines in revenuesincluding  headcount reductions in 2020. Refer to Note 17 “Restructuring” to the</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.813341</td>\n",
       "      <td>business;• manage our platform and our business assets and expenses in light of the COVID-19 pandemic and related public health measures issued by various jurisdictions,including  travel  bans,  travel  restrictions  and  shelter-in-place  orders,  as  well  as  maintain  demand  for  and  confidence  in  the  safety  of  our  platform  during  andfollowing the COVID-19 pandemic; • plan for and manage capital</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.809412</td>\n",
       "      <td>pandemic, including sales, marketing and costs relating to our efforts to mitigate the impact of the COVID-19 pandemic. Furthermore, we have expanded overtime  to include more asset-intensive offerings such as our network of Light Vehicles, Flexdrive, Lyft Rentals and Lyft Auto Care. We are also expanding the supportavailable  to drivers at our Driver Hubs, our driver-centric service centers and community spaces, Driver Centers, our vehicle service centers, Mobile Services,</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.809215</td>\n",
       "      <td>COVID-19 pandemic  in March  2020. We have adoptedmultiple  measures in response to the COVID-19 pandemic. We cannot be certain that these actions will mitigate some or all of the negative effects of the pandemic on ourbusiness.  In light of the evolving and unpredictable effects of COVID-19, we are not currently in a position to forecast the expected impact of COVID-19 on our financialand operating results in fu ture periods.Revenue Recognition Revenue</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.808421</td>\n",
       "      <td>estimated.In  response to the COVID-19 pandemic, we have adopted multiple measures, including, but not limited, to establishing new health and safety requirements forridesharing  and  updating  workplace  policies.  We  also  made  adjustments  to  our  expenses  and  cash  flow  to  correlate  with  declines  in  revenues  including  headcountreductions in 2020. 56</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "visualize_retrieved_nodes(new_nodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1dcc798a-3940-4426-8110-d9a3f7dc5a68",
   "metadata": {},
   "outputs": [],
   "source": [
    "new_nodes = get_retrieved_nodes(\n",
    "    \"What initiatives are the company focusing on independently of COVID-19?\",\n",
    "    vector_top_k=40,\n",
    "    reranker_top_n=5,\n",
    "    with_reranker=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38bae391-7dca-4f8d-a90e-ada1beddcd2e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Score</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.0</td>\n",
       "      <td>remotely, as well as permanent return to workarrangements and workplac e strategies;• the inability to achieve adherence to our internal policies and core values, including our diversity, equity and inclusion practices and initiatives;• competitive pressures to move in directions that may divert us from our mission, vision and values;• the continued challenges of a rapidly-evolving industry;• the increasing need to develop expertise in new areas of business that</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9.0</td>\n",
       "      <td>platfor m and scaled user network.Notwithstanding  the impact of COVID-19, we are continuing to invest in the future, both organically and through acquisitions of complementary businesses. Wealso  continue to invest in the expansion of our network of Light Vehicles and Lyft Autonomous, which focuses on the deployment and scaling of third-party self-drivingtechnology  on the Lyft network. Our strategy is to always be at the forefront of transportation innovation, and we believe that through these</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9.0</td>\n",
       "      <td>the transport ation needs of customers, employees and other constituents.• Grow Active Riders. We  see opportunities to continue to recoup and grow our rider base amid the continuing COVID-19 pandemic. We may make incrementalinvestments in our  brand and in growth marketing to maintain and drive increasing consumer preference for Lyft. We may also offer discounts for first-time ridersto  try Lyft or provide incentives to existing riders to encourage increased ride frequency. We</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8.0</td>\n",
       "      <td>to  grow  our  business  and  improve  ourofferings,  we will face challenges related to providing quality support services at scale. If we grow our international rider base and the number of international drivers onour  platform, our support organization will face additional challenges, including those associated with delivering support in languages other than English. Furthermore, theCOVID-19 pandemic  may impact our ability to provide effective and timely support, including as a result of a decrease in the availability of service providers and increasein</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6.0</td>\n",
       "      <td>pandemic and responsive measures;• natural disasters, economic downturns, public health crises or political crises;• general macroeconomic conditions;Operational factors • our limited operating history;• our financial performance and any inability to achieve or maintain profitability in the future;• competition in our industries;• the unpredictability of our results of operations;• uncertainty regarding the growth of the ridesharing and other markets;• our ability to attract and</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "visualize_retrieved_nodes(new_nodes)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "llama_index_v2",
   "language": "python",
   "name": "llama_index_v2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
